Non-commutativity from coarse grained classical probabilities 
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Non-commutative quantum physics at the atom scale can arise from coarse graining of a classical 
statistical ensemble at the Planck scale. Position and momentum of an isolated particle are classical 
observables which remain computable in terms of the coarse grained information. However, the 
commuting classical product of position and momentum observables is no longer defined in the 
coarse grained system, which is therefore described by incomplete statistics. The microphysical 
classical statistical ensemble at the Planck scale admits an alternative non-commuting product 
structure for position and momentum observables which is compatible with the coarse graining. 
Measurement correlations for isolated atoms are based on this non-commutative product structure. 
We present an explicit example for these ideas. It also realizes the discreteness of the spin observable 
within a microphysical classical statistical ensemble. 



The origin of the non-commuting product of operators 
is one of the old puzzles for the conceptual understand- 
ing of quantum physics. One may aim for an explanation 
of the axioms of quantum mechanics within a more gen- 
eral probabilistic formulation of the theory, based on an 
ensemble with positive probabilities and using the usual 
probabilistic relations between measurements, expectation 
values of observables and the probability distribution. It 
is often believed that such an approach must fail because 
of the commutativity of the classical correlation function 
for two observables. No go theorems using Bell's unequal- 
ities [l[ are based on the use of this classical correlation 
|2|. In this note we present an explicit example how a 
non-commuting product of observables arises from a clas- 
sical statistical ensemble. This product is appropriate for 
the description of correlations of measurements in subsys- 
tems. Non-commutativity arises from "coarse graining" or 
a reduction of the available information. All features of 
quantum physics can be described in this way, including 
interference, tunneling, or the violation of Bell's inequali- 
ties. 

We use a classical statistical ensemble with states r and 
positive probabilities p T > for the description of some 
microphysical system. As an example we take discrete 
variables at the Planck scale or beyond, which may be as- 
sociated with occupation numbers for fcrmions. Within 
this setting we want to describe a single particle, such as 
an isolated atom, in terms of an appropriate subsystem. 
We assume that the observables relevant for the atom, i.e. 
position and momentum X and P, are well defined as clas- 
sical observables for the microphysical classical statistical 
ensemble. They have fixed values X T and P T for every state 
r and their expectation values are computed according to 
the basic rules of classical statistics. Only a small part of 
the information available for the microphysical or "Planck 
scale" system will be needed and employed for the descrip- 
tion of the atom at the " Bohr scale" . Much of the informa- 
tion can be integrated out if we concentrate on the " coarse 
grained" subsystem and the details of the microphysical 
system will not be important for our setting. The concept 
of an atom as a subsystem is familiar in quantum field the- 
ory, where atoms are viewed as excitations of a complicated 



vacuum, which involves infinitely many degrees of freedom 
and collective phenomena such as spontaneous symmetry 
breaking. 

In the microphysical ensemble the classical product of 
position and momentum observables is well defined and 
commutative. For every state r the value of the observable 
X ■ P = P ■ X is given by {X ■ P) T = {P ■ X) T = X T P T . 
For our construction of the subsystem, however, this clas- 
sical product remains no longer defined. The expectation 
value of the classical product (X ■ P) can be computed from 
the information available for the microphysical system, i.e. 
from the probability distribution {p T } and the values X T 
and P T for every state r. This complete statistical informa- 
tion is no longer available for the subsystem, since a large 
part of the microphysical information is lost by the coarse 
graining. The subsystem describing the atom is character- 
ized by incomplete statistics |3| for which the expectation 
value of X ■ P is no longer computable. 

On the other hand, the microphysical ensemble admits 
also a product structure for the observables X and P 
that is different from the classical product. This product 
is not commutative. It is this non-commutative product 
that remains computable in terms of the information avail- 
able for the subsystem. We will argue that the outcome 
of good measurements of subsystem properties should be 
computable in terms of the statistical information available 
for the subsystem - otherwise one would measure "envi- 
ronment properties" beyond the properties of the isolated 
atom. Correlations between the results of such good mea- 
surements are described by the non-commutative product. 
We will show how this non-commutative product can be 
associated with the product of quantum operators for po- 
sition and momentum. 

The general concepts how incomplete statistics for sub- 
systems leads to measurement correlations based on a non- 
commutative product of observables have been discussed in 
ref. [4j, 5|. In ref. 4] it is shown in detail that these cor- 
relations indeed violate Bell's inequalities and are not in 
contradiction with the Kochen-Specker theorem [6|. What 
has been missing so far is a concrete example for a construc- 
tion where a subsystem as an atom is described in terms of 
a classical ensemble for microphysical degrees of freedom. 



This note discusses such an example, where the microphys- 
ical degrees of freedom may be associated in a vague sense 
to fermions at the Planck scale. Our aim is not in any 
way the construction of a realistic particle physics model, 
and the model is only meant as an intuitive setting for 
a microphysical ensemble. The notions of "Planck scale" 
or "Bohr scale" can be replaced by arbitrary other scales, 
and an interpretation of the microphysical degrees of free- 
dom in terms of fermions is not necessary. Our conceptual 
setting applies to a large class of possible fundamental par- 
ticle physics models - the important notions involve only 
the coarse graining to a subsystem describing a one parti- 
cle state. In principle, it can apply to arbitrary classical 
statistical systems provided that the coarse grained "one 
particle state" can be appropriately defined. In this way 
quantum features may be discovered in macroscopic clas- 
sical statistical systems. 

Usually, fundamental theories of particle physics are not 
based on a classical statistical ensemble. This holds despite 
the striking observation that the analytic continuation of 
the functional integral to euclidean space often allows for 
such a classical statistical formulation, as well known for 
lattice gauge theories. It is also well known that non- 
commuting operator structures can be defined for the clas- 
sical statistical ensemble in euclidean space. Still, because 
of no go theorems based on Bell's inequalities, it is almost 
general belief that a quantum field theory in Minkowski 
space cannot be described by a classical statistical ensem- 
ble. The present note demonstrates that this belief is un- 
founded - the road to a classical probabilistic formulation 
of a fundamental theory of particle physics is open. 

In a second part of this note we implement another im- 
portant property of quantum physics within a classical sta- 
tistical description, namely that individual measurements 
of an observable only find eigenvalues of the associated 
quantum operator. We show how this quantum postulate 
follows from the standard classical statistical setting for 
measurements for two examples of quantum operators with 
a discrete spectrum, namely local projectors and the spin 
of a particle. 

Function observables 

Let us start with a discrete chain of L points, labeled by 
s = 1 . . . L, which may be placed on a circle. To each point 
we attach an "occupation number" n s which can take the 
values one or zero, n s = 0, 1. We can associate this system 
with fermions on a one-dimensional lattice. Alternatively, 
one may interprete the n s as L bits of a computer. A state 
of the system is given by an ordered sequence of L bits, 
t = {n s }, such that the number of different states r equals 
N s = 2 L . We consider a classical statistical ensemble with 
positive and normalized probabilities p T for each state, 
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r>0, J2P- = 1 - 



(1) 



Classical observables take a fixed value A T for every state 
r and obey the standard definition for expectation values 



(A)=Y,PtA t . 



(2) 



We will be interested in classical observables that can 
be associated to functions on a circle f(x). For such an 
observable we define for each sequence {n s } — r a fixed 
function f T {x), which we normalize according to 



dxtf(x) = 1. 



(3) 



The expectation value of such a "function-observable" 
reads 



</(s)> = 5>r/r(a0- 



(4) 



We can also define observables which measure properties 
of a function, like the "roughness" 



(5) 



R T = / dx{d x f T {x))' 

or a position variable 

X T = / dxxf^(x). 



(6) 



For these classical observables the expectation values are 
always given by eq. @. Intuively, one may realize such 
a "function observable" by associating a given range in s 
to an interval in x, and taking f T (x) positive if there are 
more occupied (n s — 1) than empty (n s — 0) bits in the 
interval, negative if there are more empty bits, and f T (x) 
close to zero for approximately equal numbers of occupied 
and empty bits in the interval. 

A simple particular function-observable can be obtained 
by the following construction. We choose a discrete set of P 
equidistant points Xi on the circle, with " lattice distance" e 
and intervals I(xi) given by Xi — | < x < x% + 5, such that 
these intervals cover the whole circle and J dx = Pe. We 
are interested in situations where the number of bits L is 
much larger than P. To every bit s we associate x = xq+sS, 
with S = (P/L)e, and arbitrary Xo, e.g. xq = 0. Thus every 
bit s "belongs" to one given interval I{x{). The function 
f T (xi) is defined as 



f r (x l )=Af- 



(2n s - I), 



(7) 



s£l(xi) 



where the sum is taken over all s which belong to the in- 
terval I(xi). Thus A/" 1 / 2 f T (xi) is simply the number of 
occupied bits minus the number of empty bits within the 
interval I(xi). The normalization factor J\f is defined by 
the requirement 



or 



JdxfUx)=J2frM = h 



Xi s£l(xi) 



(8) 



(9) 



(For simplicity we may take L odd such that N > 0) 
is guaranteed.) At the end we can take the continuum 



limit e — > or P — >• oo in the standard way, with 
J dx — eJ2x ■ ■> fr( x ) = € ^ fr{xi), either with fixed S/e <C 1 
or with S/e — > 0. 

The detailed description how we associate to a sequence 
t a function f T (x) will not be important. As an alternative 
to the "stepwise function" ([7]) we may associate to each bit 
sequence r a continuous and differentiate function even 
for finite L. This could be achieved by some smoothening 
prescription of the stepwise function. One could also use 
the complete system of periodic functions on the torus 
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(10) 



and define an appropriate map from r to a sequence of 
numbers {afc, bk}- For our purpose we only need the exis- 
tence of a well defined map r — > f T (x) . 

Our setting is easily generalized to more than one di- 
mension, where x 11 may be coordinates on a d-dimensional 
torus. We can also consider more than one "species" of bits 
with occupation numbers n QiS for every s and a = 1 . . . F 
labeling different species. There are then N s = 2 FL dif- 
ferent states t = {n QjS }. The function observables fa,r{%) 
can be defined separately for each species, for example by 
using in cq. (|7J n a . s , and adapting the normalization ac- 
cording to 

Y J fdxfl T {x) = l. (11) 

For two species we can define a derivative observable 



Pt 



dx[fl, T (x)d x f 2 , T {x) - / 2j r(x)9x/l,r(x)]. (12) 



We may also introduce a complex structure with a complex 
function f T (x) — fi, T {x) + if2,r(x), such that 

P T = [ dxf*(x)(-id x )f r (x) , { dxt T {x)f T (x) = l. (13) 



The derivative observable resembles the momentum ob- 
servable in quantum mechanics. Correspondingly, the po- 
sition observable reads 



(14) 



X T = I dxf*(x)xf r (x). 



Despite the similarities with the quantum formalism we 
recall that both X T and P T take sharp values in every state 
r. The classical correlation function is commutative, 



{X ■ P) cl = (P ■ X) cl = J2ptX t Pt 



(15) 



Coarse graining 

For the computation of the expectation values of position 
(X) — ^2 T p T X T and momentum (P) = ^ T p T P T one needs 
much less information than contained in the probability 
distribution {p T } for the classical statistical ensemble. A 



specification of {p T } requires 2 2L real numbers. Many dif- 
ferent {p T } lead to the same (X) and (P). In order to 
concentrate on the relevant information for a particle sub- 
system we perform a " coarse graining" , as defined by the 
density matrix 



p{x,x') = ^2p T f T (x)f*(x'). 



(16) 



As in quantum mechanics, the density matrix is normalized 
and hermitean 

Trp = / dxp(x,x) — 1 , p*{x,x) = p(x',x). (17) 



We can associate to the observables X and P the operators 
X and P, 

X(x', x) = Six' - x)x , P(x', x) = -iS(x' -x) — , (18) 

ox 

such that the expectation values obey 

(X) = 2_,PtX t — Tr(Ap) = dxxp(x,x), 
t •* 

(P) = Y,PrPr=MPp) (19) 

r 

= —i I dx'dxS(x l — x)d x p(x,x'). 



The information contained in p is therefore sufficient for 
the computation of (X) and (P). 

The density matrix is not a property of a single state r 
but rather involves the whole probability distribution {p T }. 
In other words, the coarse graining corresponds to a map 
{Pt} —t p{x, x'). This map is not invertible - many different 
{p T } are mapped to the same p(x,x') such that a large 
part of the microphysical information is lost by the coarse 
graining. We observe that eq. (Q~9|) constitutes precisely the 
quantum rule for the computation of expectation values of 
position and momentum from a quantum density matrix. 
Indeed, p(x, x') is a positive matrix, such that together 
with eq. (fTT|) it obeys all the required properties of the 
density matrix in quantum mechanics. The positivity of 
p, i.e. J , g*(x)p(x,x')g(x') > for arbitrary g, follows 
directly from the definition (|16[) . 

Products of observables 

The operator representation of X and P allows the in- 
troduction of a product structure for observables that dif- 
fers from the classical product. This "quantum product" 
is induced by the operator product. For example, the ob- 
servable X 2 is associated to the squared operator A 2 , i.e. 

(X 2 ) = f dxx 2 p{x,x). (20) 

It can be realized as a classical observable by 

(X 2 ) T = fdxf;(x)x 2 f T (x), (21) 



such that the expectation value (|20[) obeys also the classical 
statistical rule 



<X 2 )=5>(X 2 ) T . 



(22) 



More formally, the quantum product lol^ X 2 can be 
defined as a map relating the classical observables X and 
X 2 . The particular choice (|2"Tj) represents an equivalence 
class of similar products that all result in the same quan- 
tum operator product [4|. 

In contrast, the classical product A-B of two observables 
A and B is given by (A-B) T — A T B T , such that (X -X) T = 



X 2 differs from (X 2 ) T , with 



(X-X) 



^2p T X 2 = ^2pt( / dxf*(x)xf T (x))' 



(23) 



We observe that the expectation value of the classical prod- 
uct (X ■ X) cannot be computed from the information con- 
tained in p(x,x'). It is therefore not compatible with the 
coarse graining, in contrast to the quantum product. 

The difference between the quantum and classical prod- 
ucts may be visualized by comparing the quantum disper- 
sion A 2 with the classical dispersion (Ax ) 2 , 



A 



(X 2 ) - (X) 2 , (A^ l) ) 2 = (X-X)- (X) 2 , (24) 



for an ensemble with (X) = 0. Consider a sequence r 
for which the occupied bits are concentrated in a region 
of space around x p , while the empty bits are concentrated 
around — x p , such that |/ T | 2 (x) remains an even function 
of x centered around x p and —x p . This sequence does not 
contribute to the classical dispersion, since X T = and 
therefore (X T ) 2 = 0, while it contributes to the quantum 
dispersion due to (X 2 ) T fa x 2 > 0. We notice that the 
quantum dispersion exceeds the classical dispersion, as can 
be seen from the identity 



A^-(A^ 



(cl) s2 



/ y l 



p T / dxf;(x)(x-X T ) 2 f T (x) > 0. (25) 



A detector which signals activity in a region around x p 
whenever there is a substantial imbalance between occu- 
pied and empty bits will measure the quantum dispersion. 
Only measurements with this type of detector can be de- 
scribed in terms of the reduced information of the coarse 
grained system. 

The issue of commutativity or non-commutativity always 
refers to a particular choice of a product between two ob- 
servables. We have seen that different types of products 
between observables can be defined consistently. The ap- 
propriate choice of the product depends on the particular 
setting how measurements are done [5j]. If a subsystem 
can be described by a reduced amount of information, the 
results of "good measurements" of two properties of the 
subsystem can only concern the information characterizing 
the subsystem. In other words, the statistical outcome of 
such measurements must be computable in terms of the 
information available for the subsystem. In our case, the 



information characterizing the subsystem is coded in the 
density matrix p. Position measurements which are consis- 
tent with this coarse grained system have to be associated 
to the quantum product X 2 , since the classical dispersion 
and (X ■ X) are not computable in terms of p. 

The use of the quantum product for repeated measure- 
ments of X in a one particle system has a natural inter- 
pretation. We can associate the positive and normalized 
function 

w(x) = p(x,x) =^2p T \f T (x)\ 2 , 

T 

w(x) > , / dxw(x) = 1 (26) 



with the probability to find the particle at the position x. 
Repeated measurements of the particle position have then 
the standard statistical interpretation in the coarse grained 
system, the expectation value for n measurements being 



(X n ) = / dxx n w(x). 



(27) 



In other words, the conditional probability to find in a 
second measurement the particle at x, if it has been found 
at x in the first measurement, equals one. The classical 
product X ■ X has no such simple interpretation. We could 
define a classical probability for simultaneous activity at x 
and y by 



Wcl 



(x,y)=J2Pr\fr\ 2 (x)\fr\ 2 (y) 



and express 



(X ■ X) = / dxdy xy w c i(x,y). 



(28) 



(29) 



This does not seem to be a very suitable one-particle con- 
cept. It is then not surprising that the classical product is 
not compatible with the coarse graining to a one-particle 
subsystem. 

As long as we are interested only in observables of the 
type X n we only need the information about the diago- 
nal elements p(x,x) — w(x). The expectation values of 
arbitrary observables G(X), defined by 

(G(X)) t = J ' dxf* T (x)G(x)f T (x) (30) 

can be computed in terms of w(x) as 

(G(X)) = $>r(GpO) T - J ' dxw(x)G(x) = tr(G(A». 

(31) 
Only the modulus of f T (x) is needed. This changes for 
derivative observables as the momentum P or the squared 
momentum P 2 defined by 

(P 2 ) T = I ' dxr T (x)(~d 2 x )f T (x)= [ dx\d x f T (x)\ 2 . (32) 



(Note that P 2 generalizes the roughness ([5]) to the case 
of two species. We keep a one-dimensional notation with 
obvious generalization to d dimensions.) We now need the 
off-diagonal elements of p according to 

(/>-; = 5>(P 2 ) T =tr(P 2 p) 

dxdx'5{x' -x)(-d 2 )p{x,x'). (33) 



Again, only the quantum product P 2 is compatible with 
the coarse graining, while the classical product 



{P-P)^Y,P^ P r=-J2pAj dx fr^)d X fAx)y 



(34) 



is not computable in terms of p. We conclude that the 
coarse grained system is described by "incomplete statis- 
tics" |3|: The classical product of two observables is not 
available for the subsystem. 

These observations extend to products of position and 
momentum observables. While the classical product X ■ P 
is not available for the coarse grained system, the quantum 
products defined by 



(XP) T 



dxf*(x)x(~id x )f T (x) 



obey 



(PX) T = J dxf*(x)(-id x )xf T (x) 



(XP) = tr(X Pp) , (PX) = tr(PXp). 



The quantum product is non-commutative 



XP -PX = i. 



(35) 



(36) 



(37) 



Thus our classical statistical ensemble implements Heisen- 
berg's uncertainty relation for measurements of position 
and momentum which are compatible with the coarse 
graining. For such measurements the quantum product 
appears directly in the measurement correlation for a se- 
quence of measurements of position and momentum. This 
correlation has to be based on appropriate conditional 
probabilities and differs from the classical correlation which 
would be based on the classical product X ■ P. The mea- 
surement correlation for two measurements, one of position 
and the other of momentum, is given [4j, |5| by 



(XP) m = ~((XP) + (PX)). 



(38) 



Finally, in the three dimensional case a definition of an an- 
gular momentum observable which is compatible with the 
coarse graining must be based on the quantum product 
Lk = ekijXiPj rather than involving the classical product 
Xi ■ Pfc . We conclude that all features of quantum mechan- 
ical observables are recovered for our classical statistical 
ensemble. Planck's quantum ft can be induced by chang- 
ing the units of P. 



For an appropriate time evolution of the classical prob- 
ability distribution {p T } the induced time evolution of the 
density matrix obeys the von-Neumann equation |5|. This 
can be achieved by a unitary evolution of {p T } (rotations 
of the unit vector q T with_B T = q 2 ) |7| which is compatible 
with the coarse graining Q . The detailed dynamics can be 
formulated as a fundamental equation for the time evolu- 
tion of {p T }- In turn, for a fixed time evolution of {p T \ the 
particular Hamiltonian governing the time evolution of the 
quantum subsystem can be computed. We will not discuss 
this issue in the present note and refer to refs. [a, 0|- 

Pure quantum states 

At this point a few comments concerning the role of pure 
quantum states are in order, (i) The coarse graining is 
based on the density matrix (|16p while the expectation 
value of the function observable (f(x)) given by eq. (@| 
plays no role. In general, the expectation values of X or P 
cannot be computed from (f(x)}. A pure quantum state 
obeys 



dx' p(x, x')p(x' , x") = p(x, x") 



(39) 



and allows the introduction of a quantum wave function 
ip(x) obeying 



p(x, x') = ip(x)ip*{x') 



(40) 



in the usual way. In general, ip(x) differs from (f(x)}. 

(ii) An exception is given for a pure classical state for 
which p T = except for one particular f with p^ = 1. In 
this case on has 



1>(x) = (f(x)) = Mx). 



(41) 



One may imagine a deterministic time evolution which re- 
mains within the space of pure classical states - either in a 
discrete way by "jumping" between different sequences r, 
or by extending the space of states by replacing the discrete 
labels 2n a — 1 = ±1 by cos(p a and performing continuous 
changes of the angles ip a . Such a time evolution realizes a 
unitary evolution of ^p{x) since by definition of f T (x) the 
norm is preserved. This special case can be viewed as a de- 
terministic hidden variable theory for quantum mechanics, 
with discrete or continuous hidden variables given by n atS 
or cos^q^. 

(iii) The coarse graining leading to the density matrix 
(ITBl can be performed in two steps. In a first step we con- 
sider an intermediate statistical ensemble with states char- 
acterized by functions f a (xi)- For this purpose we collect 
all bit sequences r which lead to fa,r{xi) within a given 
function-interval f a {xi) - 7 Q < f a ,r(xi) < f a {xi) + J a - 
The function intervals are chosen such that an arbitrary 
fa,r(xi) belongs precisely to one of the intervals labeled by 
fa(xi). (This should hold for all space intervals labeled by 
Xi.) We may define a collective or coarse grained probabil- 
ity 



p[f a (xi)] = Y^ 



Pt 



l/a(x 4 )i 



(42) 



G 



where the sum extends over all r for which f a , T (xi) belongs 
to the function- interval labeled by f a (Xi). In the contin- 
uum limit p[/q(x)] becomes a functional of f a (x). Many 
different sequences r can "belong" to the same function 
f a {xi) such that the mapping {p r } — > p[fa(x)] is not in- 
jective and constitutes indeed a first coarse graining step. 
The second step defines p(x, x') (for F — 2 and complex 
/) as 



p(x,x') 



f{< 



p[f(Xi)]f(Xi)f*(Xi) 



(43) 



where the sum extends over all function-intervals of the 
complex function f(xi). This equals the definition (fil)|) . In 
the continuum limit p[/(x)l is a functional of the complex 
function f(x). Again, the map pf/C^)] — > p(x,x') is not 
injective. In the language of ref. [5[ we can identify the 
collective states labeled by f{x) with the "microstates" a, 
while t labels the " substates" . 

(iv) We can define a pure state on the level of the first 
coarse graining step by p[f(xi)] = 1 for some selected func- 
tion /, while p[/(£i)] = for all other /. In this case one 
finds a pure state quantum density matrix with associated 
wave function 



i'(x) = f(x). 



(44) 



We may view p[f] as the probability to find a given possible 
wave function /. In turn, for a pure quantum state one 
particular / = ip is selected with unit probability. Mixed 
quantum states arise if p[f] differs from zero for two or 
more functions /. Inversely, it is a necessary condition for 
a pure quantum state that only one / contributes in eq. 
(|4"3"|). If one is interested only in position and momentum 
observables and their quantum products one could forget 
about the basic states r and start directly with a classical 
statistical ensemble characterized by p[f] > 0, J T>fp[f] = 
1. A pure state could be interpreted as a deterministic 
hidden variable theory with hidden variables now given by 
the real functions fi(x), f2(x). It is sufficient that / = ip 
evolves in time according to a Schrodinger equation. This 
aspect shares common features with Bohm's interpretation 
of quantum mechanics J9| or with the setting of ref. [l(| ■ 
Mixed quantum states find an interpretation in terms of 
"classical probabilities" p[f] according to eq. (|43|) . 

(v) One could be tempted to use use at the intermedi- 
ate level a classical statistical interpretation for which both 
the position and the momentum observables X and P are 
realized as classical observables. In this case they would 
have a "sharp"value X[f],P[f] for every state f(x), given 
by eqs. (Til?)) , (TT4")) with f T replaced by /. However, for a 
deterministic hidden variable theory of this type the ex- 
planation of a non-vanishing dispersion A 2 is difficult to 
understand. We will also see that simple local projection 
observables for a quantum particle cannot be implemented 
in such a setting. 

If we start from the microphysical ensemble with states r 
the status of the observables X and P for a given microstate 
f(x) is different. They are now "probabilistic observables" 



yl [ljj with a probability distribution of possible measure- 
ment values for every given f(x). Then X[f] and P[f] 
denote the means obtained from this distribution. The 
"quantum product" X 2 ,P 2 ,XP etc. differs again from 
the "classical product" at the intermediate level, which 
reads (A x B)[f] = A[f]B[f]. Also the classical product 
at the intermediate level is not compatible with the coarse 
graining (1431) (except for pure states). For example, X x X 
involves four powers of / and (X x X) cannot be computed 
from the information contained in p(x 1 x'). This contrasts 
with the quantum product X 2 . The above arguments con- 
cerning the relevance of the quantum product for position 
measurements remain valid. Again, the central ingredient 
for the appearance of non-commutativity in a classical sta- 
tistical ensemble is the presence of a product structure for 
observables that differs from the classical product and is 
appropriate for the measurements of position and momen- 
tum. 

Local projectors 

In the second part of this note we address another cru- 
cial feature of quantum physics, namely the postulate that 
individual measurements of an observable always result in 
eigenvalues of the associated operator. This is particularly 
striking for operators with a discrete spectrum as angular 
momentum. Every individual measurement can only yield 
one of its discrete eigenvalues. We want to show here how 
this postulate can follow from the standard classical statis- 
tical interpretation of measurements within our setting of 
an ensemble based on states specified by bit sequences r. 

Finding classical probabilities which reproduce for some 
given observables the same expectation values as in quan- 
tum mechanics is per se not difficult. For example, one 
may associate the states of the ensemble with /, where 
the variables / or "function variables" f(x) correspond to 
the real and imaginary parts of a quantum wave function. 
Classical observables take for each / the same expression 
as quantum observables in terms of the wave function. For 
any probability distribution p[f] the quantum expectation 
values will then be found according to the density matrix 
(|4"3")l . (Classical ensembles with less degrees of freedom have 
also been found - see, for example, ref. 12[ for integer 
spins.) However, this is not enough for an explanation of 
measurements and correlations in quantum mechanics. For 
example, for three-state or "spin-one" quantum mechanics 
a spin observable would take in every state / the value cor- 
responding to the expectation value in the associated pure 
quantum state. According to classical statistics, measure- 
ments should then find a continuity of values between —I 
and 1, rather then the discrete values —1,0, 1 predicted by 
quantum mechanics. 

In contrast, the discreteness of quantum physics can be 
realized in a straightforward way if a classical observable 
associated to a quantum operator with a discrete spectrum 
has for every state r one of the discrete values belonging 
to its spectrum. According to the basic setting of classical 
statistics each measurement finds then a value within the 
spectrum, while the probability distribution {p T } specifies 
with which probability a given eigenvalue of the associated 
operator will be found. We demonstrate this issue by dis- 
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cussing the classical statistical implementation of two fa- 
miliar quantum operators with a discrete spectrum, local 
projectors and the spin. 

Within quantum mechanics, we may define "local pro- 
jectors" or "interval observables" which multiply the wave 
function by a factor 

J(x,a) = 6 (x + | -x) 6 (x-x+ |V (45) 

They have the property that J(x,a)i/)(x) = ip{x) for 
x G I(x,a), and J(x,a)ip(x) = otherwise, with I(x,a) 
the interval x — | < a; < x + §. The corresponding op- 
erators are projectors, J 2 (x,a) = J{x,a). Therefore the 
eigenvalues of the operator J(x,a) are 1 or 0, and the in- 
terpretation is simple: either a particle detector covering 
the interval I(x, a) finds a particle (J = 1) or not (J = 0). 
The eigenstates with eigenvalue J — 1 are all ip{x) which 
vanish identically outside the interval I(x,a), while eigen- 
states with J = vanish inside the interval. We will use 
the discrete formulation of functions as used for f T (Xi) in 
eq. (0. We concentrate on intervals I(xi,e) = I(xi) of 
size e and on interval observables J(xi) = J(xi,e). The 
eigenfunctions il)t Xi \{x) of the interval observables J{xi) 
form a basis, such that arbitrary ip(x) can be written as 
ip(x) = J2 X - a {x i )'4 ! (xi){x)- They equal one inside the in- 
terval I(xi) and vanish outside, with an additional fac- 
tor e -1 ' 2 in the continuum normalization. The functions 
ipt Xi \{x) are also eigenfunctions of the position operator X, 
with eigenvalue x% . We can write 



X = 



y %i>j [%%)• 



(46) 



So far standard quantum mechanics. In order to realize 
the interval observables J(xi) as classical observables we 
need to associate to every sequence r a number (J{xi)j = 
1, 0. With respect to each J(xi) the states r decay into two 
classes, depending if \J{xij\ equals one or zero. On the 
coarse grained level the expectation value of J{xi) reads 



(J(Xi)) = p(Xi,Xi) = U>(Xi). 



(47) 



Consistency therefore requires that the assignments of 
(J(xi)j and probability distributions {p T } obey 

(J(Xi)) = J2M J (*i))r = X>#(*i)/r(*i)- (48) 



We define here f T (Xi) by eq. ([7]). The normalization (|SJ) 
implcs \f T (xi)\ 2 < 1 and therefore < J(xi) < 1, as it 
should be. However, if we admit arbitrary sequences r it 
seems difficult to find an assignment (J(xi)) such that eq. 
(|48)l holds for arbitrary probability distributions {p T }- 

At this point we recall that the quantum wave func- 
tion ip{x) is supposed to describe a one-particle-state. In 
contrast, arbitrary probability distributions {p T } describe 
states with an arbitrary particle number. We therefore 
have to select the one-particle states. They correspond to a 
particular class of probability distributions {p T }- As a first 
example we consider "locally concentrated sequences" for 



which f T (xi) vanishes except for one particular £j(t), re- 
flecting a "particle number" one at Xi , and zero elsewhere. 
For locally concentrated sequences the normalization of f T 
implies |/ r (ii(r)| 2 = 1. It is then straightforward to define 
the classical interval observables as 



(J(x t )) T ^\f T (x l )\ 2 =0,l. 



(49) 



In this case the possible values of measurements of J(xi) 
are indeed given by the values (0,1) of the classical observ- 
ables. They equal, in turn, the eigenvalues of the associ- 
ated quantum operators. Probability distributions {p T } for 
which p T differs from zero only for the locally concentrated 
sequences describe one particle states. 

The notion of one particle states can be extended beyond 
the locally concentrated sequences. It is sufficient to asso- 
ciate to each sequence r a particular space interval located 
at Xi(r), for example the one where \f T (xi)\ 2 reaches its 
maximum. We then define the interval observables by 



(j(xi(T)) T = 1 , (j( Xi # Xi{T)) T = 0. 



(50) 



The "allowed probability distributions" {p T } for a one par- 
ticle state are the ones which obey eq. (|48l) . The other 
"forbidden probability distributions" describe states with 
an admixture of contributions with total particle number 
different from one. For one particle states the total par- 
ticle number observable is trivial and may be associated 
with £ Xi (J(x i )) r = 1. 

The reader may notice that the notion of one particle 
states depends on the resolution e. A one particle state 
for resolution e remains a one particle state for resolution 
ne,n > 1, as follows from combining n intervals of size 
e. However, a one particle state obeying eq. (|48| for a 
resolution e needs not to obey similar conditions for subin- 
tervals of I{xi,e). A one particle state with resolution e 
may therefore contain multiparticle states for a resolution 
e' < e. This closely reflects known physical properties. A 
one atom state for a resolution distance exceeding suffi- 
ciently the atom size appears as a multiparticle state with 
nucleons and electrons on a smaller resolution scale. 

It is at the level of the projection observables that a de- 
terministic setting based on "hidden variables" f(x) fails to 
account for the discreteness of quantum mechanics. While 
the expectation values of X and P can be reproduced cor- 
rectly in such a setting, there seems to be no "classical" 
projection observable of the type (J(xi))(f) which asso- 
ciates to each state or function f(x) one of the two allowed 
values 1 or 0. In contrast, the microphysical ensemble with 
states r allows (J{xi))(f) to be a probabilistic observable 
which has a probability distribution of values 1 or for 
each "microstate" f(x). 

Spin 

Another characteristic discrete quantum degree of free- 
dom is the spin of a particle. For particles with internal 
degrees of freedom we can consider a higher species number 
F. For a spin one half particle we take F — 4, leading to 
two-component complex functions f T — (fi T +1/2 T , H r + 
ih, T ). Replacing f* by /t,/t/ = ff + f$ + /f + ft\ a ll 



previous formulae for position and momentum observables 
are easily generalized. We next define three two-level ob- 
servables by 



{S X ) T = 20 ( J dx(h <T {x)h, T {x)-h, T {x)U, T {x)))-l, 

(S 2 )r = 20( J dx(flA x )UA X ) ~ hA X )hA X ))) ~^ 

(5 3 ) T = 26(Jdx(fl T (x) + fl T (x) 

-fl T {x)-ft T {x)))-l. (51) 

In every state r they take one of the discrete values ±1, 
(Sfc) T = (Sk)r = 1, such that measurements of this observ- 
able should only find the values +1 or —1, according to 
the rules of classical statistics. We will identify the observ- 
ables Sk with the spin of a particle in the direction k, up 
to a normalization factor h/2. We further define local spin 
observables by 



(Sk{x l )) T = (S k ) T (j(Xi)) n 



(52) 



Their measurement can yield the values +1, —1, or 0. 

The allowed probability distributions {p T } for one par- 
ticle states with spin are restricted by the conditions 

(J(x l )) = ^Pr{J{Xi)) T = 1 ^2 l PTfl{Xi)f r {Xi), 

T T 

(Si(xi)) = ^2p T (S 1 (x 1 )) T = 2^2p T {fiA x i)faA x i) 

T T 

+hA X i)fiA X i)) 

(S2(Xi)) = ^2pr(S2(Xi)) T ^2^2p T (f 1 ^ T (xi)f 4 ,T{x i ) 

t r 

-f%A X i)hA X i))i 
(S 3 ( Xi )) = J2Pr{S 3 (x t )) T =J2pAftr(^) + flr(^) 



-fiA x i)-ftA x i))- 



(53) 



In the complex basis the generalization of the density ma- 
trix (fTB| is a complex hermitean 2x2 matrix. Eq. ([55)1 
implies the standard relation 

(S k (x)) = tr(S k (x)p) = tv(T k p{x,x)), (54) 

with operators for the local spin 

S k (x)(y, y') = r k S(x - y)S(y - y'). (55) 

Similarly, the spin observables obey 

(S k ) = tr(S k p) , S k (y,y') = nS(y -y') = / dxS k (x). 



(56) 
As it should be, the product of two different spin oper- 
ators is not commutative, and we can define the associ- 
ated quantum product for the classical spin observables 
(S k ) T . Again, the classical product of spin observables, 



(S k ■ Si) T — (S k ) T (Si) T , cannot be computed from the in- 
formation available at the coarse grained level and we en- 
counter incomplete statistics. 

We can express the expectation values (l53l) in terms of 
the functions f a (xi) defined on an intermediate level of 
coarse graining using eq. (j42|) . 



(57) 



(J(xi)} = I Vfp[f}J( Xi )[f}, 
(S k ( Xi )) [vfplf] S k (xi )[/"], 



with 



J( x i)[f] 
Si(xi)[f] 

S2(Xi)[f] 

S 3 (x l )[f] 



- ft 



2(fi(xi)f 3 (x 1 ) + f 2 (x i )U(x i )) 

P{Xi)Tlf{Xi), 

2(fl(Xi)f4(Xi) - f2(Xi)f 3 (Xi)) 
P{Xi)Tlf{Xi), 

fH^)T 3 f(x t ). (58) 



The complex two-component function f(x) has all the 
properties of the quantum wave function for a spin one 
half particle. In particular, we notice the identity S\{xi) + 
5|(xi) -I- Sl(xi) — J 2 {xi) for arbitrary /, and the normal- 
ization of / related to the identity J2 X J( x i) = 1- 

For a microphysical classical statistical ensemble which 
describes an isolated particle the "one particle condition" 
(1581) must be preserved by the time evolution of the prob- 
ability distribution {p T }- The normalization of / implies 
that such a time evolution describes a generalized rotation 
in the real Hilbert space spanned by the functions f a (xi). 
If the time evolution remains compatible with the complex 
structure this transfers to a unitary transformation in a 
complex Hilbert space. The infinitesimal unitary transfor- 
mations define the hermitean Hamiltonian for the quantum 
time evolution. (If for suitable subensembles the average 
values of J(xi),S k (xi) and a local momentum observable 
P(xi) obey an appropriate "purity constraint" [J,0] we can 
actually use their mean values in the subensemble in order 
to define the functions f a {xi) by eq. (|58[). This issue will be 
discussed in a separate paper. An independent definition 
of the function observables f T {xi) is no longer necessary in 
this setting. For the description of an isolated particle it 
is then sufficient that the purity constraint is preserved by 
the time evolution.) 

At this stage we have implemented the quantum spin ob- 
servable S k as classical observables that take values +1 or 
— 1 for every classical state r. This has to be generalized for 
arbitrary directions of the spin observables. This general- 
ization requires that the action of rotations of the spin can 
be implemented on the level of the probability distributions 
{p T } such that eqs. (|57l) . (|58]l hold for spins S u in arbi- 
trary directions, with a suitable assignment (S^),- = ±1. 
An explicit construction how the spin rotations are imple- 
mented on the level of classical probability distributions 
can be found in ref. [5j. For continuous rotations an in- 
finite number of states r is required, N s — > oo, while for 



finite N s only a discrete subgroup of the rotations can be 
realized. This does not matter in practice, since even for 
finite S equal to the Planck length and V some atomic vol- 
ume the number of states N s = 2 4V / S is extremely high. 
Similar constructions can realize the angular momentum 
observables Lk as discrete classical observables with fixed 
values (Lk) T which belong to the spectrum of the quantum 
operators Lk- 

In conclusion, we have presented an explicit classical sta- 
tistical ensemble for which a one-particle state can be de- 
fined for a subclass of probability distributions {p T }. All 
properties of an isolated particle can be expressed in terms 
of the coarse grained information contained in a density 
matrix p(x,x') which is computable form the probability 
distribution {j4 }. One-particle observables as position, 
momentum or spin are realized as standard classical ob- 
servables in the microphysical classical ensemble. How- 



ever, their classical products cannot be defined in terms 
of p(x,x') - the coarse grained system is described by in- 
complete statistics. On the other hand, our system al- 
lows the definition of a non-commutative quantum product 
for these observables which remains compatible with the 
coarse graining. This quantum product provides for the 
correlations of measurements of properties of an isolated 
particle. We demonstrate explicitly how the quantum for- 
malism with non-commuting operators emerges from our 
classical statistical description. While on the microphys- 
ical level both the classical and the quantum product of 
the one-particle observables can be defined, only the quan- 
tum product "survives" the coarse graining. In this sense 
the coarse graining of the information for subsystems is 
the origin of the non-commutative structure of quantum 
physics. 
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